With the aid of methods for analyzing temporal sequences of digital images, the aim is to use one or more temporal sequences of digital images, possibly with the aid of measurement data of further sensors, such as tachometers or range finders, for example, to determine and analyze the movements of moving objects in space, the intrinsic movement of the imaging sensors (cameras) recording the images, and the spatial structure of the scenes visible in the image sequences.
Such methods are required in various technical fields, for example, in information technology, communication technology and automation technology, in order reliably to detect moving objects, determine their form, classify them with the aid of their movement, and mutually to coordinate the movement of a plurality of automatic systems. In this case, there is a requirement for as wide ranging an immunity as possible against external disturbances, and for a reliable distinction between changes in illumination and movements.
Examples of applications of such methods are found in video-based systems for monitoring and control functions, for example in production engineering or in road traffic control and instrumentation (intelligent traffic light control). The determination of spatial structures and the analysis of spatial movements is of the highest significance for applications in robotics, as well as for aims in autonomous navigation. For the purpose of supporting vehicle drivers, there is a need for systems which are capable, with the aid of one or more video cameras and of the vehicle speed determined by the tachometer, and with the aid of other data such as measured distance data, for example, of detecting moving objects in the environment of the vehicle, the spatial structure of the vehicle environment and the intrinsic movement of the vehicle in the environment, and of tracking the movement of detected objects. Finally, in communication technology the reduction of image data for purposes of transmission and storage of image data is steadily gaining in significance. Precisely in the case of coding temporal image sequences, analysis of movements delivers the key to a decisive reduction in datasets or data rates (movement compensation, model-based image coding).
Descriptions of different methods for analyzing temporal sequences of digital images are found in the specialist literature. The simplest type of method is represented by the so-called change detection (J. Wiklund, G. Granlund, "Image Sequence Analysis for Object Tracking", Proceedings of the 5th Scand. Conference on Image Analysis, Stockholm 1987), in which temporally successive images are compared. An improved change detection is described, for example, in A. Mecocci, Moving Object Recognition and Classification in Natural "Environments", Signal Processing 18 (1989), pages 183-194. It is based on the comparison of a temporal image sequence with a sequence of reference images which are calculated in a temporally recursive fashion and from which the moving objects are eliminated. The essential disadvantage of change detection is to be seen in that a stationary camera is presupposed.
Methods which are based on estimating displacement vectors or displacement vector fields in the image plane are of general applicability for analyzing movements. Such methods are, for example, described in H. H. Nagel, "Analyse und Interpretation von Bildfolgen", ("Analysis and Interpretation of Image Sequences") Informatikspektrum (1985) 8: pages 178 to 200 and pages 312 to 327, or in J. K. Aggarwal, N. Nandhakumar, "On the computation of Motion from Sequences of Images--A Review", Proceedings of the IEEE, Vol. 76, No. 8, 1988. In these types of method, displacement vectors of moving objects or of features such as edges or corners, for example, on moving objects are determined in the image plane.
Methods on the basis of two-dimensional displacement vectors or displacement vector fields are basically suitable for analyzing movements in digital image sequences. Their applicability is not subject to any sort of limiting assumptions concerning the movement of the camera or the movement and number of the moving objects visible in the image. However, with regard to analyzing the movements in three-dimensional space, these methods are to be regarded only as preprocessing or measurement methods, since the three-dimensional structure of the environment and the three-dimensional movement of objects cannot be directly gathered from two-dimensional displacement vectors.